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Abstract 

In recent studies, new measures of complexity for nonlinear systems have been 
proposed based on probabilistic grounds, as the LMC measure (Phys. Lett. A 
209 (1995) 321) or the SDL measure (Phys. Rev. E 59 (1999) 2). All these 
measures share an intuitive consideration: complexity seems to emerge in nature 
close to instability points, as for example the phase transition points characteristic 
of critical phenomena. Here we discuss these measures and their reliability for 
detecting complexity close to critical points in complex systems composed of many 
interacting units. Both a two-dimensional spatially extended problem (the 2D Ising 
model) and a oo-dimensional (random graph) model (random Boolean networks) 
are analysed . It is shown that the LMC and the SDL measures can be easily 
generalized to extended systems but fails to detect real complexity. 
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1 Introduction 



The quantitative characterization of complexity in nature has received considerable at- 
tention over the last decade [1-4]. Although there is no universal agreement about the 
definition of what complexity is, increasing evidence suggests that it often emerges close 
to marginal instability points of some kind. Critical phase transitions are a particularly 
well studied example [5]. In all these systems, some generic properties like self-similar 
structures, 1/f noise and maximum information transfer seem to emerge in a rather 
spontaneous way. 

In order to quantify the degree of complexity in a system, many measures have been 
proposed [1,6-9]. Comparative classifications have been made based on general character- 
istics like the type of partitions in phase space, statistical structure, etc [10,11]. In spite 
of the large number of proposed measures of complexity, it seems to be a matter of fact 
that these measures have to share some stringent conditions. 

2 The complexity measure LMC 

Recently, Lopez-Ruiz, Mancini and Calbet (LMC) presented a new measure based on 
a simple set of assumptions [12]. In short, let us assume that an arbitrary dynamical 
system has N accesible states which belong to a set S M = {xf^\ i = 1, N} and have an 
associated probability distribution 

n M = {pi(ji) = P[x = Xi (ti)];i = 1, N} (1) 

Here \x stands for a given parameter which allows to describe the transition from the 
ordered (low /i) to the disordered (high jx) regimes. As usual, Y^L\ -^HaO = 1 and Pi(fx) > 
Mi = 1, N. The LMC measure is based on a combination of two different quantities: 
(a) the Boltzmann entropy, 

N 

H^P^-Eft-MkSftM (2) 

i=i 

where = (pi(/i), ...,pjv(//)) and (b) the so-called disequilibrium V^, defined as: 

v,*P*) = jb(pM-^) 2 (3) 

Using (2) and (3) the LMC measure of complexity is defined by the functional: 

C M (P M ) = H^PJV^) (4) 

The basic idea involves the interplay of two different tendencies: the increase of entropy 
as the system becomes more and more disordered and the decrease in as the system 
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approaches equiprobability/disorder. The previous definition can be easily generalized to 
the continuum, i. e. : 

C M = Hf/D^ = ~(^f P»( x ) ^gp^(x) dx^j ^ J pl(x) dx^j (5) 

This measure gives small values for highly homogeneous (ordered) or heterogeneous (dis- 
ordered) states but it must reach a maximum at some intermediate value //*. The ap- 
plication of this measure to some well known nonlinear dynamical systems described 
by iterative maps gave consistent results. For example, the study of the logistic map 
x n +i = (J>Xn(l — x n ) showed that C M undergoes a rapid increase for (/i — /i c ) — > close to 
the critical point defining the onset of the period-three window. 




Figure 1: Entropy (open circles) and disequilibrium (open squares) for a single spin from 
a 40 x 40 Ising model. In all figures concerning this model, r = 2 x 10 4 transients were 
discarded and averages were performed over T = 10 4 time steps (Glauber dynamics). 
We can see that both quantities reach their extreme values near T = T c (= 2.27). Inset: 
1-spin LMC complexity (eq. 12) computed from the previous sets of values. A maximum 
is obtained at T = T c with a sharp decay to zero for T > T c 

But there is no guarantee that the maximum of C M will occur at the appropiate fi*. 
In fact, the LMC measure can be written as a combination of various ifg-Renyi entropies 
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[4] of a probability measure r\. This seems reasonable, but there are too many possible 
ansatzes which would do the same. These problems have been recently discussed [13] in 
relation with the LMC measure. In this context, Feldman and Crutchfield [13] have shown 
that the LMC measure is neither an intensive nor an extensive thermodynamic variable. 
Another problem in the previous definition is the specific form of V^. The equiprobability 
is a consequence of an underlying assumption of weak or no interaction. Such distribution 
can be reached from a variational argument. Specifically, the variation of the functional: 



N 



o 



(6) 



leads to pj = 1/N and so a maximum H M = logiV. But if other constraints play a role 
(as it occurs in far-from-equilibrium systems) then disorder does imply equiprobability. 
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Figure 2: Average fluctuation of the magnetization M, defined from <r(M) = (\M(t) — 
(M)\). A maximum is obtained at T = T c , when fluctuations at all scales are observed. 
Three snapshots of a 2D Ising simulation are shown for three different temperatures. 
Inset: distance to independence for two nearest spins, as calculated from (18). We can 
see that it behaves closely to a(M) 

Let us first show that LMClike measures can be easily extended to complex, spatially 
extended systems. The presence of correlations/interactions in a given system can be mea- 
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sured through the spatial correlation function [14]. For a given sequence S = s±, S2, ■ ■ ■ , sn 
when St G £ = {a a } (with i — 1, . . . , k) it is given by 



r(*) = EE a « a ^(*)-fE^ 



) 



2 



(7) 




Two-point correlation functions are used in statistical physics as a quantitative charac- 
terization of phase transitions. In the thermodynamic limit T(S) is expected to diverge 
at criticality or reach a maximum if the system size is finite. For one of these systems, 
both temporal and spatial patterns contain information about the existence of correlations 
arising from the local interactions. 

As a standard example, let us consider the 2D Ising model. In this paper all of 
our results for the Ising model are obtained with a L = 40 lattice and using Glauber 
dynamics. This example allows us to consider a binary (N = 2) set of states and so the 
relation between H M and is guaranteed to be univocal [12]. Here \i is the temperature T 
and the set of possible states for each unit is simply = {?>!}> where the arrows stands 
for spin up and down, respectively. The mutual information M(S) for two (^-neighbors 
spins clearly fulfil the requeriment of being low at the ordered and chaotic regimes and 
maximum at the transition point. We have: 



Now at T < T c most of the spins are in the same direction (say up) so P^ ps 1 and 
P i = p. = p(|) « 1 and so M(5) w 0. Here 5 ab is the Dirac delta. When T > T c we have 
P(i,j) ~ PiPj and so M(5) — > as T grows. Information transfer becomes maximum at 
T c [4] as a consequence of long-range correlations. In fact it can be shown that -for binary 
sequences- a formal relation between the correlation function and the mutual information 
can be derived [14]: 



which leads to the conclusion that M(S) decay to zero at a faster rate than T(5). So 
if T(5) w 5 _/3 then M(S) ps 5~ 2/3 . Beyond these specific relations, any properly defined 
complexity measure should give equivalent results. 

Clearly, if only a single spin is used, we can write the previous definitions for the 
disequilibrium and entropy as 






(9) 
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D T (P(V) = P(T) 



2 



+ (i-P(T)) 



2 



(10) 



H T (P(W = -P(t) logP(T) - ((1 - P(t)) log(l - P(|)) 



(11) 
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and the LMC measure will be given by 

Cr(P(T)) = Dt(P(WHt(P(W (12) 

Our results are shown in figure (1). We can see that Ht grows fast when approaching 
T c = 2.27 from T < T c . This is an expected result as far as < M(T = T c ) >= and so 
on the average -P(t) = 1/2. On the other hand if we take D T we can see that it drops to 
zero at T c , as expected. The LMC measure is shown in the inset. We see a fast growth 
of Ct with a sharp decay at the critical point. So this characterization gives nearly null 
complexity for T > T c . This result, however, does not correspond with the analysis of the 
model by means of standard techniques. As an example, we show in figure 2 the average 
fluctuation of the magnetization for the Ising model. It shows a maximum at T c and a 
decay for T > T c . Such decay is not as sharp as the one given from the LMC measure, 
and it measures in fact the existence of correlations for T > T c which are not taken into 
account by the 1-spin LMC quantity. 



3 LMC extended and distance 
to independence 

The LMC measure can be extended by considering the statistics of interacting units. The 
reason of this choice is clear: long range correlations are created through interactions 
between nearest spins, as defined by the Ising Hamiltonian Hi sing = — JSiSj, being J 
the coupling constant. A first possibility is to use the statistics of K-spin blocks. The 
joint entropy for a K— block spin system is defined as 

h t° = - 2 2-2 P(M2,...,ijr)logp(ii,i 2 ,...,ix-) (13) 
and the corresponding disequilibrium will be given by 

d? } = - E 2-2 Uii,i2,..,i K )-^] 2 (i4) 

For K = 2, two given spins (here we take two nearest neighbors) are considered and 
a set of joint probability measures can be used pij = P[s a = i; Sf> = j] , i.e., the joint 
probability of having the a-spin in state i and the 6-spin in state j. Again, we have a 
normalization condition J^Pij — 1- Because introduces correlations in a very general 
way, we can redefine the disequilibrium in terms of these values as: 

^'=2 2 (pa-\) (15) 
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and the joint entropy as: 

Ht=~ E E Pi3 l °gPij ( 16 ) 

These are simple extensions of the previous LMC approach but now the interaction (and 
so the intrinsic correlations) between parts of the system are taken into account. In figure 
3 the previous quantities are shown. We can clearly observe that both quantities behave 
smoothly close to T c . This behavior leads to an LMC-complexity that shows again a 
maximum at T c but with a slower decay for T > T c . This is consistent with our explicit 
consideration of correlations into the joint probabilities. 




' 0.0 1.0 2.0 3.0 4.0 5.0 

Temperature 



Figure 3: Joint entropy (16) (open squares) and disequilibrium (15) (filled triangles) for 
the 2D Ising model. Inset: corresponding LMC complexity (product of Joint entropy and 
disequilibrium). Now a maximum at T c is also obtained but correlations on both sides 
are detected (compare with figure 1, inset) 

An important drawback of these measures, as pointed before, is the definition of the 
disequilibrium based on the distance to equal probabilities. We should remind that for 
the 2.D-Ising model the entropy raises sharply close to T c towards its maximum value, But 
this is not the case in most problems. A simple generalization can be obtained by taking 
into account a much more natural measure, i. e. the distance to independence defined, 
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for a K-block as: 



4 X) = - E E ••• E fp(ii,i2,...,<K)-np(^)) 



(17) 



iie{T,l}i 2 e{T,l} ijce{U} 
which, for K = 2 simply reads 



E E (Py-PiP,-) 2 (18) 
<e{T,l}j6{T,i} 

Now, V l j is a measure of how far are the subsystems from being independent (i.e., from the 
identity pij = PiPj). The distance to independence Dtp, does not behave as Dj.. It can be 
easily shown that the distance to independence acts similarly to the joint information (8) 
at both extremes (where total disorder leads to = PiPj or when a complete homogeneous 
distribution leads to pa = piPi and zero for the other cases j ^ i). In fact: 



M = EPO- log ^ = - E^ log f 1 + ™>-J*i\ 



(19) 



approximating to second order the logarithm, we have: 



i.e, the distance of the independence is a non-weighted approximation to the joint infor- 
mation. 

The behavior of this measure for the 2D Ising model is shown in the inset of figure 2. 
We see that Dip behaves as the fluctuations in the magnetization. This measure is able to 
detect the onset of complex fluctuations (and the underlying correlations at many scales) 
and decays for T > T c in the same way as the average fluctuation. So in fact itself 
seems to be a simple and consistent measure of complexity very close to the underlying 
physics. As we will see below, this measure is also consistent in other cases. 



4 Testing LMC and SDL whit 
Random Boolean Networks 

The previous results might suggest that the generalized LMC complexity is an appropiate 
measure of correlations. This intuition, however, is not supported from the analysis of 
other complex systems. As an example, let us consider the dynamics of a random Boolean 
network (RBN) [15]. This model is defined by a set of discrete maps: 

S i (t + l)=A i (S il ,S ia ,...,S iK ) (21) 
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Figure 4: Distance to independence, D^, as a function of the bias p, for RBN's of K = 3 
and N = 250 spins, p has been varied in intervals of size Ap = 10~ 2 . After a transient of 
length t = 250 is discarded, we measure over 250 time steps. Each point is the average 
over 400 samples. The theoretical analysis gives a critical point at p c = 0.79 (derived 
from (20)) which is revealed by the distance to independence (21). The snapshots show 
particular examples of the dynamics for p = 0.6 (chaotic regime), p = 0.79 (transition 
point) and p = 0.9 (ordered regime). Inset: the corresponding LMC complexities for the 
single-spin (open triangles) and two-spin (black circles) measures under identities compute 
conditions. A maximum at p ~ 0.69 is obtained 



where S; 6 E = {0,1}; i = 1,2,..,N and A^S^, Si 2 , Si K ) is a Boolean function ran- 
domly choosen from the set Tk, of all the Boolean functions with K variables. So each 
spin in (21) receives exactly K inputs from a set of randomly choosen neighbors. This 
is no longer a finite-dimensional extended system. Here the dynamics takes place on a 
random directed graph. In fact, the statistical properties of this model can be understood 
in terms of damage spreading on a Bethe lattice [16]. 

It is well known that RBNs show a phase transition at some critical points. Specifically, 
if p = PfA^S^, Si 2 , Si K ) = 1] (i. e. p is the bias in the sampling of Boolean functions) 
a phase transition curve in the (K,p) plane is shown to exist [16-19] and is given by 

K = r (22) 

2p(l-p) 1 ; 

The analysis of this system shows that at the critical point percolation of damage takes 
place for the first time i. e. a single flip of a binary unit can generate an avalanche of 
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changes through the whole system and appropiate order parameters for the (second order) 
phase transition can be defined. 

As argued in the previous example, maximum structure and correlation should be 
expected at the critical point, and so maximum complexity. In figure 4 we show the 
behavior of D 1 as a function of the bias p for a K = 3 RBN. We can see a maximum 
at p c = 0.79, as predicted by the critical condition given by (22). So the distance to the 
independence shows a maximum at the critical point, as expected from a properly defined 
complexity measure. Here D 1 has been computed by averaging 

Di = ( E (Pij-P^Pi) 2 ) (23) 

where < ... >at stands for average over all the units. We can similarly define the corre- 
sponding single-spin and two-spin LMC measures for the RBN and the results are also 
shown in the inset of figure 4. Althought both LMC measures have a maximum at a 
given 0.5 < p* < 1.0, they fail to detect the critical point. Here p* « 0.69p c . This 
is understandable as far as the p parameter strongly influences (a priori) the statistical 
distribution of state frequencies. 

In [20], Shiner, Davison and Landsberg introduce a new measure of complexity (here- 
after SDL measure): 

C aP = H a (l - Hf (24) 

where H is the normalized Boltzmann entropy. The SDL measure also tries to satisfy 
the criteria of maximum complexity between order and disorder as in LMC (in fact the 
LMC measure can be viewed as an approximation to the SDL as shown in [20]). H is 
interpreted as a measure of disorder and 1 — H as a measure of order. The parametrization 
with a and (3 allow to fit ad hoc according to the specific cases considered. 

The SDL measure has been criticized in [21]: the SDL measure give the same value 
for systems structurally different but with identical H . This problem is severe in RBN's. 
The entropy H in RBN's depends, exclusively on the bias p, thus nets with differents con- 
nectivities (and then with different structural dynamic) but identical bias show identical 
H and then identical C curves. Since the maxima of complexity depend too on K, the 
SDL measure will fail: we can always fit a and (5 in different ways for each connectivity 
K in such a way that we can recover the maximum. The value of the normalized entropy 
H that maximize the SDL measure is a /(a + ($) [20]. But it is easy to see from (22) and 
fixed K, that: 

H = -p c \og Pc - (1 - p c ) log(l - p c ) = a/ (a + (3) (25) 

allows an infinite number solutions of values for a and (3, which is far from satisfactory. 
Similarly, we can define H in blocks, but the problem persists. 
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5 DISCUSSION 



In summary, we have analysed the validity of the LMC approach as an effective mea- 
sure of complexity for systems composed by many units in interaction. Two different 
(although standard) problems have been considered. For the 2D Ising model, where a 
phase transition is known to occur at the Curie temperature, the LMC measures showed 
a maximum close to T = T c . A measure based on the statistics from a single spin failed, 
however, to detect correlations for T > T c although such correlations exist. The rea- 
son was the lack of information about interactions between nearest spins and the fact 
that -P(T) —> 1/2 as the critical temperature is approached from below. This problem was 
solved by considering an extension of the LMC measure to a joint probability distribution. 
Such extension was able to show a maximum at T c and well-defined corrlations. A simple 
extension of the disequilibrium function was proposed -the distance to independence D 1 - 
as an alternative measure of complexity. This quantity (which is in fact a second-order 
mutual Renyi information) was shown to consistently detect the critical point for the 2D 
Ising model and RBNs with a lower computational effort than LMC complexity. A direct 
extension of the LMC measure to the random Boolean network model was shown to fail 
in detecting maximum complexity at the critical point. The reason for this result is clear 
is we consider that both the entropy and the disequilibrium change smoothly with the p 
parameter. Such parameter strongly influences the statistics of the numbers of units in 
one of the two states, and in so doing it definitely influences the values of both functions. 
Although correlations can be in principle detected, the relevance of p in defining a priori 
the values of entropies and disequilibrium is very important. These results confirm the 
intuition that the LMC and SML measures (and similar quantities) will fail to detect real 
complexity. 
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